Face authentication system and face authentication method

ABSTRACT

A face authentication system includes a registration device that acquires a face image for registration of a user in a facility and specific information used for specifying the user, one or more face authentication machines that acquire a face image for authentication of the user captured when the event is generated, a face management server that is connected to the registration device via a network, and accumulates the face image for registration and the specific information in association with each other, and a face collation server that generates face feature data from the face image for registration and the face image for authentication, and performs collation processing between pieces of the face feature data.

TECHNICAL FIELD

The present disclosure relates to a face authentication system thatperforms face authentication processing based on image data obtained bycapturing a target person and a face authentication method.

BACKGROUND ART

In the related art, a system has been developed in which faceauthentication of a target person is applied to open/close control of agate through which the target person passes. For example, a faceauthentication system is known (refer to PTL 1) that includes a readerwhich receives an identifier transmitted from a wireless tag (remotetype IC card) possessed by the user, an acquirer which acquires a facefeature amount registered corresponding to the identifier received fromthe wireless tag, an imaging portion which images the user, an extractorwhich extracts the face feature amount from the image data imaged by theimaging portion, a face collator that collates whether the face featureamount extracted by the extractor match the face feature amount acquiredby the acquirer, and an open/close controller which opens or closes anexit of a gate according to the result of collation by the facecollator.

CITATION LIST Patent Literature

-   PTL 1: International Publication No. 2018/181968

SUMMARY OF THE INVENTION

Incidentally, in the above technique in the related art, although it ispossible to narrow down the target person of face authentication (thatis, registered face feature amount) by the identifier transmitted fromthe wireless tag of the user, since it is necessary for the user whopasses through the gate to carry the IC card at all times, it isdifficult to safely manage personal information (that is, informationthat can identify a specific individual) contained in the IC card. Inaddition, in the above technique in the related art, no consideration isgiven to a mechanism in which both face image data and face feature dataare placed on a cloud server or the like and the data is safely managed.

Furthermore, in the facility, various events (for example, payment ofvalue for services received in the facility, purchased items, and thelike) may occur not only by the user passing through the gate but alsoby the actions of the user in the facility. However, in the abovetechnique in the related art, no consideration is given to theapplication of face authentication to various other events.

Therefore, a main object of the present disclosure is to provide a faceauthentication system that can easily perform face authentication of auser for each event which may occur in a facility while safely managingpersonal information of the user who uses the facility and a faceauthentication method.

The face authentication system of the present disclosure that registersa face image of a user in a facility and performs face authenticationfor each event generated due to an action of the user in the facility,the system including a registration device that is installed in thefacility and acquires a face image for registration of the user andspecific information other than the face image, the specific informationbeing used for specifying the user; one or more face authenticationmachines that acquire a face image for authentication of the usercaptured when the event is generated; a face management server that isconnected to the registration device via a network and accumulates theface image for registration and the specific information in associationwith each other; and a face collation server that generates face featuredata from the face image for registration and the face image forauthentication, and performs collation processing between pieces of theface feature data.

In addition, the face authentication method of the present disclosurethat registers a face image of a user in a facility and performs faceauthentication for each event generated due to an action of the user inthe facility, the method including acquiring a face image forregistration of the user and specific information other than the faceimage, the specific information being used for specifying the user inthe facility; acquiring a face image for authentication of the usercaptured when the event is generated; accumulating the face image forregistration and the specific information in association with each otheron a face management server; and generating face feature data from theface image for registration and the face image for authentication, andperforming collation processing between pieces of the face feature databy a face collation server.

According to the present disclosure, it is possible to easily performthe face authentication of the user for each event that may occur in thefacility while safely managing the personal information of the user whouses the facility.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an overall configuration diagram of a face authenticationsystem according to the present exemplary embodiment.

FIG. 2 is an explanatory diagram illustrating an example of an eventoccurring in a facility.

FIG. 3 is an explanatory diagram illustrating an outline of a collationgroup.

FIG. 4A is an explanatory diagram illustrating an outline of an inquiryfor face collation performed by face authentication machine 1.

FIG. 4B is an explanatory diagram illustrating an outline of the inquiryfor face collation performed by face authentication machine 1.

FIG. 4C is an explanatory diagram illustrating an outline of the inquiryfor face collation performed by face authentication machine 1.

FIG. 5A is an explanatory diagram illustrating an outline of a facecollation unit.

FIG. 5B is an explanatory diagram illustrating an outline of the facecollation unit.

FIG. 5C is an explanatory diagram illustrating an outline of the facecollation unit.

FIG. 6 is a block diagram illustrating a schematic configuration of faceauthentication machine 1.

FIG. 7 is a block diagram illustrating a schematic configuration ofmanagement terminal 2.

FIG. 8 is a block diagram illustrating a schematic configuration ofregistration device 4.

FIG. 9 is a block diagram illustrating a schematic configuration of facemanagement server 5.

FIG. 10 is a block diagram illustrating a schematic configuration offace collation server 6.

FIG. 11 is a block diagram illustrating a schematic configuration of anexternal device.

FIG. 12 is an explanatory diagram illustrating an outline of face imageextraction processing.

FIG. 13 is an explanatory diagram illustrating an outline of an inquiryfor face authentication using collation group information.

FIG. 14 is an explanatory diagram illustrating an example of aregistration screen displayed on registration device 4.

FIG. 15 is a sequence diagram illustrating a procedure of registrationprocessing for a user.

FIG. 16 is a sequence diagram illustrating a first example of faceauthentication processing for the user.

FIG. 17 is a sequence diagram illustrating a second example of faceauthentication processing for the user.

FIG. 18A is an explanatory diagram illustrating an example of anauthentication screen displayed on face authentication machine 1.

FIG. 18B is an explanatory diagram illustrating an example of anauthentication screen displayed on face authentication machine 1.

FIG. 18C is an explanatory diagram illustrating an example of anauthentication screen displayed on face authentication machine 1.

FIG. 19 is a sequence diagram illustrating a third example of faceauthentication processing for the user.

FIG. 20 is an explanatory diagram illustrating a login screen displayedon management terminal 2.

FIG. 21A is a screen displayed on management terminal 2 at the time ofregistration, reference, update, and deletion related to the associationbetween face authentication machine 1 and the collation group.

FIG. 21B is a screen displayed on management terminal 2 at the time ofregistration, reference, update, and deletion related to the associationbetween face authentication machine 1 and the collation group.

FIG. 22 is a screen displayed on management terminal 2 at the time ofregistration, reference, update, and deletion related to the associationbetween face authentication machine 1 and the collation group.

FIG. 23 is an explanatory diagram illustrating an example of amanagement screen of the user displayed on management terminal 2.

FIG. 24 is an explanatory diagram illustrating an example of aconference room search screen displayed on management terminal 2.

FIG. 25 is an explanatory diagram illustrating an example of an attendeedisplay screen displayed on management terminal 2.

FIG. 26 is an explanatory diagram illustrating an example of a personalconnection search screen displayed on management terminal 2.

DESCRIPTION OF EMBODIMENT

A first disclosure made to solve the problem is a face authenticationsystem that registers a face image of a user in a facility and performsface authentication for each event generated due to an action of theuser in the facility, the system including a registration device that isinstalled in the facility and acquires a face image for registration ofthe user and specific information other than the face image, thespecific information being used for specifying the user; one or moreface authentication machines that acquire a face image forauthentication of the user captured when the event is generated; a facemanagement server that is connected to the registration device via anetwork and accumulates the face image for registration and the specificinformation in association with each other; and a face collation serverthat generates face feature data from the face image for registrationand the face image for authentication, and performs collation processingbetween pieces of the face feature data.

Accordingly, the face image for registration of the user and thespecific information are acquired in the facility, the face image forregistration and the specific information in association with each otherare accumulated on the face management server, and when the event isgenerated, the collation processing using the accumulated face image forregistration is performed by the face collation server. Therefore, it ispossible to easily perform the face authentication of the user for eachevent that may occur in the facility while safely managing the personalinformation of the user who uses the facility.

In addition, in a second disclosure, one of the events is generated byan action of passing through a gate device that manages passage in thefacility, by the user, and at least one of the face authenticationmachines cooperates with the gate device.

Accordingly, when the user passes through the gate device in thefacility, the face authentication of the user can be easily performed.

In addition, in a third disclosure, one of the events is generated by anaction of passing through a door that manages an entry into apredetermined room in the facility, by the user, and at least one of theface authentication machines cooperates with a lock device for the door.

Accordingly, when the user enters the predetermined room in the facility(for example, conference room or room staying for a certain period oftime), the face authentication of the user can be easily performed.

In addition, in a fourth disclosure, the event is generated due to anaction of payment by the user via a payment terminal device used in thefacility, and at least one of the face authentication machinescooperates with the payment terminal device.

Accordingly, when the user performs the payment procedure in thefacility, the face authentication of the user can be easily performed.

In addition, in a fifth disclosure, the registration device acquiresinformation displayed on a business card possessed by the user as thespecific information.

Accordingly, the information of the face image of the user and theinformation displayed on the business card of the user are associatedwith each other and accumulated, so that it is possible to easilyacquire the information used to specify the user who is the target ofthe face authentication.

In addition, in a sixth disclosure, the registration device acquiresinformation displayed on a payment card possessed by the user as thespecific information.

Accordingly, the information of the face image of the user and theinformation displayed on the payment card of the user (for example,credit card) are associated with each other and accumulated, so that itis possible to easily acquire the information used to specify the userwho is the target of the face authentication.

In addition, in a seventh disclosure, the registration device acquiresinformation indicating a place of stay of the user in the facility asthe specific information.

Accordingly, the information of the face image of the user and theinformation indicating the place of stay of the user (for example, roomnumber or area number) in the facility are associated with each otherand accumulated, so that it is possible to easily acquire theinformation used to specify the user who is the target of the faceauthentication.

In addition, an eighth disclosure is a face authentication method thatregisters a face image of a user in a facility and performs faceauthentication for each event generated due to an action of the user inthe facility, the method including acquiring a face image forregistration of the user and specific information other than the faceimage, the specific information being used for specifying the user inthe facility; acquiring a face image for authentication of the usercaptured when the event is generated; accumulating the face image forregistration and the specific information in association with each otheron a face management server; and generating face feature data from theface image for registration and the face image for authentication, andperforming collation processing between pieces of the face feature databy a face collation server.

Accordingly, the face image for registration of the user and thespecific information are acquired in the facility, the face image forregistration and the specific information in association with each otherare accumulated on the face management server, and when the event isgenerated, the collation processing using the accumulated face image forregistration is performed by the face collation server. Therefore, it ispossible to easily perform the face authentication of the user for eachevent that may occur in the facility while safely managing the personalinformation of the user who uses the facility.

Hereinafter, an exemplary embodiment of the present disclosure will bedescribed with reference to the drawings.

FIG. 1 is an overall configuration diagram of a face authenticationsystem according to the present exemplary embodiment.

This face authentication system is provided with face authenticationmachine 1, management terminal 2, face authentication server 3 (cloudserver), and registration device 4 (face authentication registrationdevice). Face authentication server 3 is provided with face managementserver 5 (face authentication management server) and face collationserver 6.

Face authentication machine 1, management terminal 2, registrationdevice 4, face management server 5, and face collation server 6 areconnected to each other via a network such as the Internet. The faceauthentication system is constructed for each provider (businessoperator) of the face authentication service. A plurality of faceauthentication machines 1 are installed at each place where the faceauthentication is required, such as a building entrance and exit or aroom entrance and exit. Registration device 4 is disposed at a receptioncounter or the like that accepts a visitor to the facility. A requirednumber of face collation servers 6 are provided according to the numberof face authentication machines 1 and the like.

Face authentication machine 1 is provided with camera 11, and camera 11acquires a captured image of a user (a visitor of a facility, a workerat a company, a store, or the like in the facility). In addition, faceauthentication machine 1 includes display 12, and display 12 displaysthe face authentication result acquired from face collation server 6 andnotifies the user of the face authentication result.

Management terminal 2 is operated by an administrator of the faceauthentication system, is configured to include a personal computer(PC), and a management application that manages the operations of faceauthentication machine 1, registration device 4, face management server5, and face collation server 6 is installed. The management applicationenables the administrator to perform various management tasks. Themanagement application is implemented as a Web application. As for theadministrator, a plurality of administrator groups having differentaccess authority for each information in the face authentication systemare set.

Registration device 4 acquires information used for processing the faceauthentication of the user (hereinafter, referred to as authenticationinformation). The authentication information may include not only a faceimage for registration of each user but also specific information thatis information other than the face image, the specific information beingused for specifying the user.

As the specific information, information displayed on the business cardof the user, information displayed on the payment card (for example,credit card or cash card) possessed by the user, information indicatingthe place of stay of the user in the facility, and the like can be used.The specific information may include information that can permanentlyspecify the user, such as the name of the user. However, the specificinformation may be information that can temporarily specify the user,such as a user number given to the user of the facility, the number ofthe conference room (including any space where an interview is possible)used by the user in the facility, and the number of the room in whichthe user stays at the accommodation facility, for example.

Face management server 5 unitarily manages user information (includingpersonal information). Specifically, face management server 5accumulates face image data for registration and specific informationfor each user in association with each other, and manages the face imagedata and specific information. Face management server 5 acquires a faceimage for registration and specific information from registration device4 at the time of user registration. In addition, face management server5 may acquire at least a part of the information included in the faceimage for registration and the specific information from managementterminal 2 (that is, administrator).

At the time of face authentication, face collation server 6 acquires theface image data for authentication of the target person of the faceauthentication from face authentication machine 1, generates facefeature data of the target person from the face image data, performsface collation by comparing the face feature data of the target personwith the face feature data of the registrant (registered user) stored ina master device, and performs the face authentication for determiningwhether or not the target person is the registrant.

In addition, face collation server 6 acquires the face image data foruser registration from face management server 5 (or registration device4) at the time of user registration prior to the face authentication,and generates the face feature data of the user from the face image datato store the face feature data on the master device. In some cases, facecollation server 6 can also acquire a captured image of the user frommanagement terminal 2 (that is, administrator) and acquire the faceimage data from the captured image.

In the present exemplary embodiment, although face management server 5(face image manager) and face collation server 6 (face image collator)are assumed to exist in physically different information processingdevices, face management server 5 and face collation server 6 may existin a single information processing device.

In addition, in the present exemplary embodiment, although managementterminal 2 and face management server 5 are provided, managementterminal 2 and face management server 5 can be configured as a singleinformation processing device. For example, by installing the managementapplication on face management server 5, face management server 5 canalso serve as management terminal 2. In addition, an administrator withlimited access authority to the information of the face authenticationsystem (for example, worker at a company or store in the facility) canaccess face management server 5 from the own information processingterminal (PC, tablet, and the like) and refer to information that isless necessary to be kept secret (for example, information on theconference that the user attended described later, information on thepersonal connections of the user, and the like) than the personalinformation of the user.

In addition, in the present exemplary embodiment, although the facefeature amount collation is performed, the face collation is not limitedto the face feature amount collation, and a collation method to whichmachine learning or the like is applied may be adopted. Furthermore, thepresent exemplary embodiment can be applied to biometric authenticationother than the face authentication.

Next, an example of an event generated by the action of the user in thefacility to which the face authentication system is applied will bedescribed. FIG. 2 is an explanatory diagram illustrating an example ofan event generated in a facility.

As illustrated in FIG. 2, user 10 in the facility can perform anoperation of registration as a user in the facility (that is,registration of information used for face collation processing) byregistration device 4 disposed at the reception. For registered userswho revisit the facility within a predetermined period, registration byregistration device 4 can be omitted.

At the facility, the face authentication is performed for each of thevarious events (refer to inside the two-dot chain line in FIG. 2)generated due to the action of people (including the registered user)within the facility.

For example, registered user 10 can pass through the security gate byreceiving the face authentication in front of the security gate thatmanages the passage of user 10 in the facility. Face authenticationmachine 1 that cooperates with gate device (an example of an externaldevice) 16 (that is, controls an open and closed state of a door) can beinstalled in the security gate.

In addition, for example, registered user 10 can enter the conferenceroom by receiving the face authentication in front of the conferenceroom in the facility. In front of the conference room, faceauthentication machine 1 that cooperates with a lock device that locksthe door of the conference room (that is, controls the locking orunlocking of the door) can be disposed. For example, in an accommodationfacility such as a hotel, instead of the conference room, faceauthentication machine 1 that cooperates with a lock device for the doorof the room in which the user stays can be installed.

In addition, for example, registered user 10 can automatically settle ausage fee of a restaurant by receiving the face authentication at therestaurant in the facility. Camera 11 capable of capturing the face ofuser 10 is installed in the restaurant, and face authentication machine1 cooperates with a payment terminal device (or payment system) used forpayment of the usage fee at the restaurant.

In addition, for example, registered user 10 can automatically settle ausage fee of a gymnasium by receiving the face authentication at thegymnasium in the facility. Camera 11 capable of capturing the face ofthe user is installed in the gymnasium, and face authentication machine1 cooperates with the payment terminal device (or payment system) usedfor payment of the usage fee at the gymnasium.

The events generated in the facility are not limited to the aboveexamples, and may include various events such as the settlement of ausage fee in a parking lot in the facility, the settlement of a usagefee of a laundry service in the facility, and the settlement of a usagefee of a bathhouse in the facility, for example.

In addition, for example, in a case where an employee of a company thatprovides services in the facility is registered as a user, by arrangingface authentication machine 1 at the entrance and exit of the backyardof the facility (for example, warehouse, workshop, and the like), it ispossible to ensure the security in the backyard.

Next, a collation group will be described. FIG. 3 is an explanatorydiagram illustrating an outline of the collation group.

In the present exemplary embodiment, the users are divided into groupsand a collation group is set for each user according to the installationlocation of face authentication machine 1 for which the user receivesthe face authentication. In addition, similarly to the user, faceauthentication machines 1 are divided into groups according to theinstallation location of face authentication machine 1, and a collationgroup is set for each face authentication machine 1. Furthermore, facecollation servers 6 corresponding to face authentication machine 1, thatis, face collation servers 6 that receive an inquiry for face collation(request) from face authentication machine 1, are divided into groupssimilarly to face authentication machine 1, and a collation group is setfor each face collation server 6.

For example, in face authentication machine 1 installed at the entranceof a multi-tenant building in which a plurality of companies areoccupying, users belonging to the plurality of companies receive theface authentication. On the other hand, information of users who belongto different organizations cannot be managed in the same database.Therefore, in such a case, a collation group for each of the pluralityof companies is formed, user registration information is stored for eachcollation group, and face authentication machine 1 performs an inquiryfor face collation to face collation server 6 of the collation group foreach company.

The user can be authenticated only by face authentication machine 1corresponding to the collation group to which the user belongs.Therefore, even a registered user cannot be authenticated by faceauthentication machine 1 that does not belong to his or her collationgroup, for example, face authentication machine 1 of a building that heor she is not allowed to enter. For example, in a case where a userworking at a certain business office goes to another business office,the user cannot be authenticated by face authentication machine 1 ofthat business office.

Next, an inquiry for face collation from face authentication machine 1to face collation server 6 will be described. FIGS. 4A, 4B, and 4C areexplanatory diagrams illustrating an outline of the inquiry for facecollation performed by face authentication machine 1.

Face authentication machine 1 performs the inquiry for face collation toface collation server 6 belonging to the same collation group as themaster device, and sends a request for face collation (processingrequest for face authentication) to face collation server 6 which is aninquiry destination (request destination). In addition, faceauthentication machine 1 holds a network address (IP address) of facecollation server 6 as the collation group information, and performs theinquiry for face collation to face collation server 6 corresponding tothe master device based on the network address.

An inquiry method for face authentication includes a simultaneousinquiry illustrated in FIG. 4A, a random inquiry illustrated in FIG. 4B,and a sequential inquiry (inquiry destination switching) illustrated inFIG. 4C.

As illustrated in FIG. 4A, in the case of simultaneous inquiry, faceauthentication machine 1 simultaneously performs the inquiry for facecollation to face collation server 6 corresponding to each of theplurality of collation groups. For example, in a case where faceauthentication machine 1 is installed at the entrance of a multi-tenantbuilding and users of the plurality of collation groups receive the faceauthentication, face authentication machine 1 simultaneously sends arequest for face collation to face collation server 6 of each group.

In order to reduce the load on one face collation server 6, it is alsopossible to divide and store the face feature data of the user belongingto one collation group in a plurality of face collation servers 6. Alsoin this case, face authentication machine 1 simultaneously performs theinquiry for face collation to all face collation servers 6 havingdifferent registered contents in the same collation group as itself.

As illustrated in FIG. 4B, in the case of a random inquiry, faceauthentication machine 1 randomly selects face collation server 6 as aninquiry destination for face collation from the plurality of facecollation servers 6 in the same collation group as itself. Theregistered contents of the plurality of face collation servers 6 are thesame as each other. That is, the same user is targeted for collation,and the face feature data of the same user is stored. In such an inquirymethod, since the inquiry from face authentication machine 1 isdistributed to each face collation server 6, the load on each facecollation server 6 can be reduced and the load can be distributed.

As illustrated in FIG. 4C, in the case of sequential inquiry, faceauthentication machine 1 sequentially selects the inquiry destinationfor face collation from the plurality of face collation servers 6 in thesame collation group as itself. Specifically, in a case where an order(priority) is given to face collation server 6, face authenticationmachine 1 selects face collation server 6 according to the order, andthere is no response from the high-ranking face collation server 6, adestination is switched to face collation server 6. That is, faceauthentication machine 1 selects first-ranked face collation server 6 inthe collation group and requests face collation server 6 for faceauthentication. In a case where there is no response from face collationserver 6, face authentication machine 1 selects another face collationserver 6 having the next rank in the same collation group, and requestsface collation server 6 for the face authentication. Redundancy (backup)can be achieved by such control.

Next, a face collation unit will be described. FIGS. 5A, 5B, and 5C areexplanatory diagrams illustrating an outline of the face collation unit.

Face collation server 6 may be provided with a plurality of facecollation units for performing the face collation. The face collationunit has a face collation process that performs a face collationprogram, and a face collation database in which information (facefeature data) of users to be compared for the face collation isregistered.

The face collation unit is provided for each collation group, andperforms the face collation in response to a request for face collationfrom face authentication machine 1 belonging to the collation group.Therefore, one face collation server 6 can handle a plurality ofcollation groups.

Here, face authentication machine 1 holds the network address (IPaddress) of face collation server 6 as the collation group information,and can perform the inquiry for face collation to face collation server6 corresponding to the master device based on the network address.Furthermore, in the present exemplary embodiment, face authenticationmachine 1 holds the network address (IP address) for each face collationunit, and can perform the inquiry for face collation to the facecollation unit corresponding to the master device based on the networkaddress.

In addition, since the face collation database is provided for eachcollation group, the face feature data of the user can be stored foreach collation group. The face collation process performs collationprocessing between the face feature data stored for each group and theface feature data generated from the face image data acquired from faceauthentication machine 1 of the same group.

Here, there are various aspects of the correspondence between faceauthentication machine 1 and the face collation unit.

In an example illustrated in FIG. 5A, face collation server 6 isprovided with one face collation unit of the collation group to whichface authentication machine 1 belongs, and face authentication machine 1and face collation server 6 correspond to each other by one-to-onecorrespondence.

In an example illustrated in FIG. 5B, although face authenticationmachine 1 and face collation server 6 correspond to each other byone-to-one correspondence, face collation server 6 is provided with twoface collation units of the collation group to which face authenticationmachine 1 belongs. In this case, in the case of face authenticationmachine 1 installed at the entrance of a multi-tenant building,information of users (for example, user A and user B) who belong todifferent organizations can be managed in different collation databases.

In an example illustrated in FIG. 5C, face authentication machine 1 andface collation server 6 correspond to each other by one-to-twocorrespondence, and face authentication machine 1 performs the inquiryfor face collation (collation request for user) to two face collationservers 6. In this case, by providing two or more same face collationunits, that is, face collation units that perform the face collationusing the face feature data of the same user, on different facecollation servers 6, it is possible to distribute the load of facecollation server 6 and deal with life and death.

Next, a schematic configuration of face authentication machine 1 will bedescribed. FIG. 6 is a block diagram illustrating a schematicconfiguration of face authentication machine 1.

Face authentication machine 1 is provided with camera 11 (face imageacquirer), display 12 (display unit), communicator 13 (transmitter andreceiver), storage unit 14, controller 15, and control signaltransmitter. 17.

Camera 11 normally captures a predetermined capturing area, when aperson enters the capturing area, the person is captured, and a capturedimage of a target person for the face authentication (includingregistered user) can be acquired. In order to save power, a motionsensor (not illustrated) may be provided to detect the arrival of aperson and activate camera 11.

Display 12 displays the progress of face authentication and the faceauthentication result, and the person who receives the faceauthentication can confirm the progress and the face authenticationresult. A speaker may be provided as an output unit for outputting theauthentication result of the target person, and the face authenticationresult may be notified by voice. In addition, display 12 can be omitteddepending on the installation location of face authentication machine 1(for example, in a case where face authentication machine 1 is installedat the security gate).

Communicator 13 communicates with face collation server 6 via thenetwork. In the present exemplary embodiment, the face image data istransmitted to face collation server 6. In addition, the authenticationresult of the user is received from face collation server 6. Inaddition, communicator 13 communicates with face management server 5 viathe network. In the present exemplary embodiment, the collation groupinformation and the like are received from face management server 5.

Storage unit 14 stores data of a captured image (face image) of theuser, collation group information, a control program performed by aprocessor constituting controller 15, and the like.

Controller 15 is provided with activation processor 21, face imageextractor 22, face collation requestor 23, authentication resultnotifier 24, authentication result adjustor 25 (history informationgenerator), and operation status monitor 26. Controller 15 includes aprocessor, and each part of controller 15 is realized by performing theprogram stored in storage unit 14 by the processor.

When face authentication machine 1 is activated, activation processor 21acquires the collation group information from face management server 5and stores the collation group information in storage unit 14. Theacquisition of the collation group information is not limited to this,and face management server 5 may distribute the collation groupinformation to target face authentication machine 1 as the collationgroup information is updated.

Face image extractor 22 can acquire a captured image of a target personfor the face authentication from camera 11 (camera image capture),detect a person's face from the captured image (face detection),determine whether the detected face size is appropriate (face sizecheck), cut out a face region from the captured image (face cutout), andacquire data on the face image (face image for authentication) of thetarget person. Although the face image data may be data of only theimage of the face region, the face image data may be a combination ofthe data of the captured image (image of a predetermined capturing area)and the position information (face frame information) of the face regionon the data of the captured image.

Face collation requestor 23 transmits a request for face collation bycommunicator 13 to face collation server 6 whose master device and thecollation group match.

Authentication result notifier 24 notifies the user of the faceauthentication result by displaying the face authentication resultacquired from face collation server 6 by communicator 13 on display 12.

Authentication result adjustor 25 controls to cooperate with externaldevice 16 based on the face collation result of the user acquired fromface collation server 6. As external device 16, for example, a gatedevice (refer to FIG. 2) that manages the passage of the user in thefacility, a lock device for the door that manages an entry and exit ofthe conference room and the predetermined area in the facility, and apayment terminal device for performing payments for services andpurchased items received by users in the facility can be used. Inaddition, in a case where authentication result adjustor 25 cooperateswith the lock device for the door of the conference room, authenticationresult adjustor 25 can generate information on the entry history inwhich a plurality of the users who enter the conference room areassociated with each other.

Face authentication machine 1 and external device 16 are directlyconnected by a communication cable or the like, or are communicablyconnected via a known network. Authentication result adjustor 25 cannotify the server or the like of the external system including externaldevice 16 of the face collation result of the user by being configuredto include the connection application.

Operation status monitor 26 monitors the operation status of the masterdevice and notifies face management server 5 of the operation status ofthe master device.

Control signal transmitter 17 transmits a control signal for controllingthe operation to external device 16.

Next, management terminal 2 will be described. FIG. 7 is a block diagramillustrating a schematic configuration of management terminal 2.

Management terminal 2 is provided with communicator 31, display 32(display unit), input device 33 (operator), storage unit 34, andcontroller 35.

Communicator 31 communicates with face management server 5 via thenetwork. In the present exemplary embodiment, screen information and thelike are received from face management server 5, and administratoroperation information and the like corresponding to the screeninformation and the like are transmitted to face management server 5.

Display 32 displays various screens. Input device 33 is a mouse, akeyboard, or the like, and operates the screen displayed on display 32.

Storage unit 34 stores a program (management application) or the likeperformed by the processor constituting controller 35.

Controller 35 is provided with graphical user interface (GUI) controller38. Controller 35 includes a processor, and each part of controller 35is realized by performing a program (management application) stored instorage unit 34 on the processor.

GUI controller 38 displays various operation screens distributed fromface management server 5 on display 32. In addition, the inputinformation is acquired and the screen is controlled according to theinput operation of the administrator using input device 33. In presentexemplary embodiment, GUI controller 38 controls the display and inputof a screen related to login, specifically, a login screen. In addition,GUI controller 38 controls the display and input on screens related touser management, specifically, screens related to registration(individual registration and batch registration), reference, update, anddeletion of user information. In addition, GUI controller 38 controlsthe display and input on screens related to collation group management,specifically, screens related to registration (individual registrationand batch registration), reference, update, and deletion of thecollation group. In addition, GUI controller 38 controls the display andinput on screens related to authentication machine management,specifically, screens related to registration, reference, update, anddeletion of the association between the authentication machine and thecollation group. In addition, GUI controller 38 controls the display andinput of the reference screen of the authentication log (historyinformation on face authentication).

Next, a schematic configuration of registration device 4 will bedescribed. FIG. 8 is a block diagram illustrating the schematicconfiguration of registration device 4.

Registration device 4 is provided with face capturing camera 41A,information acquisition camera 41B (personal information acquirer),display 42 (display unit), communicator 43, storage unit 44, andcontroller 45.

Face capturing camera 41A can acquire a captured image of the user bycapturing the face of the user stopped by at the reception in thefacility, for example.

Information acquisition camera 41B can acquire a captured image of abusiness card by capturing the business card presented by the user.However, information acquisition camera 41B can capture not only abusiness card but also belongings of the user (for example, a card forpayment by the user, a card with information on the accommodation of theuser, and the like) on which information capable of specifying the useris displayed and acquire the captured image. Registration device 4 canbe provided with a placing table (placing surface) 116 (refer to (C) inFIG. 14) on which a user can easily place a business card or the like inthe capturing area of camera 41B in order to reliably capture images byinformation acquisition camera 41B.

In registration device 4, one camera may be used to configure cameras41A and 41B for face capturing and information acquisition. In addition,the capturing timing of face capturing camera 41A and informationacquisition camera 41B can be determined by the operation of the user(for example, pressing the capturing button) in registration device 4.Alternatively, a receptionist in the facility may decide the capturingtiming.

Furthermore, registration device 4 can be provided with an informationreading device (for example, RFID reader) capable of reading thespecific information of the user by wireless communication from aninformation recording medium (for example, RFID tag) possessed by theuser.

Display 42 can display the registration procedure (refer to (A) to (F)in FIG. 14) of the authentication information (including the personalinformation of the user) to the user. In addition, by attaching aspeaker to display 42, the registration procedure and the like may beguided to the user by voice. In addition, by configuring display 42 witha touch panel, it is possible to acquire the information input by theuser by touch operation.

Communicator 43 communicates with face management server 5 via thenetwork. In the present exemplary embodiment, the face image data andthe specific information are transmitted to face management server 5.

Storage unit 44 stores data of the captured image (face image) of theuser, the specific information of the user, the control programperformed by a processor constituting controller 45, and the like.

Controller 45 is provided with face image extractor 46, specificinformation acquirer 47, registration requestor 48, GUI controller 49,and operation status monitor 50. Controller 45 includes a processor, andeach part of controller 45 is realized by performing a program stored instorage unit 44 by the processor.

Face image extractor 46 can acquire a captured image of a target personfor face authentication from face capturing camera 41A, detect aperson's face from the captured image, and determine whether or not aface image of the user is appropriately acquired. In addition, in a casewhere the acquired face image is not appropriate, face image extractor46 can display a message prompting the user to re-capture on display 42.Face image extractor 46 may have functions of camera image capture, facedetection, face size check, and face cutout, similarly to face imageextractor 22 of face authentication machine 1 described above.

Specific information acquirer 47 has an optical character recognition(OCR) function, and generates specific information (here, business carddescription information) from characters and symbols extracted from acaptured image by information acquisition camera 41B.

Registration requestor 48 transmits a user registration request to facemanagement server 5 by communicator 43.

GUI controller 49 displays various guidance screens for the userdistributed from face management server 5 on display 42. In addition,the input information is acquired and the screen is controlled accordingto the input operation of the user using the input function of display42 (here, touch panel).

Operation status monitor 50 monitors the operation status of the masterdevice and notifies face management server 5 of the operation status ofthe master device.

Next, a schematic configuration of face management server 5 will bedescribed. FIG. 9 is a block diagram illustrating the schematicconfiguration of face management server 5.

Face management server 5 is provided with communicator 51, storage unit52, and controller 53.

Communicator 51 communicates with management terminal 2 via the network.In addition, communicator 51 communicates with face authenticationmachine 1 via the network. In addition, communicator 51 communicateswith registration device 4. In addition, communicator 51 communicateswith face collation server 6.

Storage unit 52 stores a face information database, a database relatedto the association information between face authentication machine 1 andthe collation group, a database related to the association informationbetween face collation server 6 and the collation group, anadministrator access log, and a control program performed by theprocessors constituting controller 53.

In the face information database, face images for registration areaccumulated as information on each registered user. In addition, in theface information database, specific information of each user isaccumulated in association with the face image for registration. Inaddition, a collation group and the like are registered in the faceinformation database. The face image of the user and specificinformation may be stored in an encrypted state for privacy protection.Furthermore, the face information database may store a part of the faceimage of the user and specific information in a state of being replacedwith anonymized information.

Controller 53 is provided with administrator access manager 61, usermanager 62, collation group manager 63, device manager 64, facecollation server manager 65, face information manager 66, databasemanager 67, operation status monitor 68, authentication log presenter69, encryptor 70, anonymized information generator 80, and imagegenerator 90. Controller 53 includes a processor, and each part ofcontroller 53 is realized by performing a program stored in storage unit52 by the processor. Each part of controller 53 is configured as a webapplication programming interface (Web API).

Administrator access manager 61 permits or denies the access (login) ofthe administrator according to the access authority of the administratorwho accesses the master device from management terminal 2. Administratoraccess manager 61 monitors the status of access (login) from managementterminal 2 to face management server 5 and face collation server 6, andwhen an access from management terminal 2 is detected, recordsinformation on the access (accessed administrator, date and time, andthe like) as an administrator access log (history information). As aresult, face management server 5 provides the administrator access logto management terminal 2 in response to the request for reference of theadministrator access log from management terminal 2, and theadministrator can view the administrator access log.

In addition, administrator access manager 61 manages access to themaster device from management terminal 2, and when the administratoroperates management terminal 2 to access the master device, informationat that time is accumulated on storage unit 52 as an administratoraccess log (history information). In addition, the administrator accesslog is presented to management terminal 2 in response to the referencerequest from management terminal 2.

User manager 62 manages information on the user such as a face image forregistration and specific information, and performs necessary processingfor the user in response to a request from management terminal 2. In thepresent exemplary embodiment, management terminal 2 requestsregistration, reference, update, and deletion for the user, and usermanager 62 performs necessary processing in response to the request. Inaddition, user manager 62 manages information on the conference attendedby the user and information on the personal connections of the user.

Collation group manager 63 manages information on the collation group,and performs necessary processing related to the collation group inresponse to a request from management terminal 2. In the presentexemplary embodiment, management terminal 2 requests registration,reference, update, and deletion of the collation group, and collationgroup manager 63 performs necessary processing in response to therequest. In addition, the collation group information, that is,information required for face authentication machine 1 to request theface authentication to face collation server 6 that matches the owncollation group is generated for each face authentication machine 1, andthe collation group information is provided to face authenticationmachine 1.

Device manager 64 manages information on face authentication machine 1and registration device 4, and performs necessary processing related toface authentication machine 1 and registration device 4 in response to arequest from management terminal 2. In the present exemplary embodiment,management terminal 2 requests registration, reference, update, anddeletion related to the association between face authentication machine1 and the collation group, and device manager 64 performs necessaryprocessing in response to the request.

Face collation server manager 65 manages information on face collationserver 6 and performs necessary processing related to face collationserver 6 in response to a request from management terminal 2. In thepresent exemplary embodiment, management terminal 2 requestsregistration, reference, and deletion related to the association betweenface collation server 6 and the collation group, and face collationserver manager 65 performs necessary processing in response to therequest.

Face information manager 66 synchronizes face information so that theface information of the user (face image, specific information, and thelike) stored in the master device and the face information of the user(face feature data of the user) stored in face collation server 6 aremaintained in a consistent state. In addition, face information manager66 copies the face information (face feature data of the user).

Database manager 67 manages the database provided in the master device,and backs up and restores the database.

Operation status monitor 68 monitors the operation status of the masterdevice, receives notification of the operation status from faceauthentication machine 1, registration device 4, and face collationserver 6, and displays the operation status of face authenticationmachine 1, registration device 4, the master device (face managementserver 5), and face collation server 6 on the screen of managementterminal 2 in response to operation of the administrator on managementterminal 2.

Authentication log presenter 69 acquires the authentication log fromface collation server 6 in response to the reference request frommanagement terminal 2, and presents the authentication log to managementterminal 2.

Encryptor 70 can perform encryption processing on at least a part of theface image of the user and specific information. The encryptedinformation is stored in storage unit 52. Face management server 5 maydelete original information (information before encryption processing)corresponding to the information encrypted by encryptor 70 from storageunit 52.

Anonymized information generator 80 can perform processing ofanonymizing at least a part of the face image of the user and specificinformation. As the anonymization processing, anonymized informationgenerator 80 can replace a part of the face image of the user andspecific information with dummy information, for example. In addition,in the anonymization processing, the dummy information may be used forinformation missing in the face image of the user and specificinformation (for example, information refused to be provided by theuser). The anonymized information is stored in storage unit 52.Encryptor 70 may encrypt only the original information (informationbefore anonymization processing) corresponding to the informationanonymized by anonymized information generator 80.

Image generator 90 generates a management screen for the administratorto confirm or change the information management status by user manager62 and device manager 64. The management screen is distributed tomanagement terminal 2 and the like.

Next, a schematic configuration of face collation server 6 will bedescribed. FIG. 10 is a block diagram illustrating the schematicconfiguration of face collation server 6.

Face collation server 6 is provided with communicator 71, storage unit72, and controller 73.

Communicator 71 communicates with face authentication machine 1 via thenetwork. In the present exemplary embodiment, face image data and thelike are received from face authentication machine 1. In addition, theauthentication result of the user and the like are transmitted to faceauthentication machine 1. In addition, communicator 71 communicates withface management server 5 via the network. In the present exemplaryembodiment, requests for various processing are received from facemanagement server 5, and responses corresponding to the requests aretransmitted to face management server 5.

Storage unit 72 stores the face collation database, the associationinformation between the user and the feature amount, the information onthe collation group of the master device, the authentication log, thecontrol program performed by the processor constituting controller 73,and the like.

In the face collation database, user face feature data and the like areregistered as information on each registered user. In addition, the facecollation database is provided for each collation group, and the facefeature data of the user is stored for each group. Here, the facefeature data of the registered user may be backed up and stored in anon-volatile memory such as a hard disk drive (HDD) and a solid statedrive (SSD) provided in face collation server 6 in preparation forunexpected disappearance from the face collation database.

Controller 73 is provided with collation group manager 81, image qualitychecker 82, face image extractor 83, face feature amount generator 84,face feature amount manager 85, face feature amount collator 86,authentication log manager 87, database manager 88, and operation statusmonitor 89. Controller 73 includes a processor, and each part ofcontroller 73 is realized by performing a program stored in storage unit72 by the processor. Each part of controller 73 is configured as WebAPI.

Collation group manager 81 manages the collation group to which themaster device belongs, and performs registration or deletion processingrelated to the collation group in response to the request from facemanagement server 5.

Image quality checker 82 determines whether or not the image of the faceregion in the captured image satisfies a predetermined quality.Specifically, the presence or absence of wearing a mask and the presenceor absence of wearing sunglasses are detected from a target image, andthe face authentication suitability (evaluation value based on faceorientation and facial expression) is calculated.

Face image extractor 83 extracts a face image from the captured image ofthe user acquired by registration device 4 at the time of userregistration. At this time, if necessary, face image extractor 83detects a face of a person from the captured image (face detection),determines whether the detected face size is appropriate (face sizecheck), cuts out a face region from the captured image (face cutout),and acquires a face image of the person. Face image extractor 83 canalso extract a face image from the captured image of the target personacquired by face authentication machine 1 at the time of faceauthentication.

At the time of user registration and face authentication, face featureamount generator 84 detects face feature points from the data of theface image for registration and the face image for authentication andgenerates face feature data, respectively.

At the time of user registration, face feature amount manager 85registers the face feature data of the user generated by face featureamount generator 84 on the face collation database corresponding to thecollation group of the user. In addition, when updating or deleting, theface feature data registered on the face collation database is deletedin response to a request from face management server 5. In addition, ina case where the version of the program related to the face featureamount generation and the face collation algorithm is upgraded, the facefeature data registered on the face collation database is updated so asto correspond to the new program in response to the request from facemanagement server 5.

At the time of face authentication, face feature amount collator 86compares the face feature data of the target person generated from theface image data for authentication acquired from face authenticationmachine 1 with the face feature data of the registrant (registered user)stored in the master device, and determines whether or not the targetperson is the registrant. Face feature amount collator 86 corresponds toa collation group associated with the master device, and in a case wherea plurality of collation groups are associated with the master device, aplurality of face feature amount collators 86 for each collation groupare included. Face feature amount collator 86 (face collation process)constitutes the face collation unit in combination with the facecollation database.

Face feature amount collator 86 calculates the degree of similarity(collation score) between the target person and the registrant. Bycomparing this similarity with a predetermined threshold value, thesuccess or failure of face authentication can be determined. In additionto notifying face authentication machine 1 of the success or failuredetermination result as a collation result, face authentication machine1 may be notified of the person ID and the degree of similarity of theregistrant having a high degree of similarity as the collation result.

When face authentication machine 1 requests the master device for faceauthentication, authentication log manager 87 accumulates theinformation at that time, that is, information such as the device ID offace authentication machine 1 as a request source for the faceauthentication and the face collation result on storage unit 72 as anauthentication log. In addition, the authentication log is provided toface management server 5 in response to the request from face managementserver 5.

Database manager 88 manages the database provided in the master device,and backs up and restores the database.

Operation status monitor 89 monitors the operation status of the masterdevice and notifies face management server 5 of the operation status ofthe master device.

In the present exemplary embodiment, each of the functions of face imageextraction, face feature amount generation, and face feature amountcollation is provided as the main functions of face collation server 6,and these functions can be configured by different informationprocessing devices that are independent of each other. For example, theface image extraction function may be configured by another informationprocessing device independent of other functions such as face featureamount generation and face feature amount collation.

Next, a schematic configuration of external device 16 will be described.FIG. 11 is a block diagram illustrating the schematic configuration ofexternal device 16.

External device 16 is provided with control signal receiver 91, driver92, power supplier 93, and controller 94.

Control signal receiver 91 receives the control signal (operationcommand from face authentication machine 1 cooperating with externaldevice 16) transmitted from control signal transmitter 17 of faceauthentication machine 1.

Driver 92 is controlled by controller 94 based on the control signalfrom face authentication machine 1. For example, in a case whereexternal device 16 is a gate device, driver 92 supplies power foropening (or closing) the door of the gate device. In addition, in a casewhere external device 16 is a lock device for the door, driver 92supplies power for locking (or unlocking) the lock device. In the casewhere external device 16 is a device that does not have a portion driven(for example, payment terminal device), driver 92 can be omitted.

Power supplier 93 supplies electric power to each part of externaldevice 16. In addition, power supplier 93 is electrically connected topower feeder 97 of face authentication machine 1, and can supplyelectric power to power feeder 97 as well. Power feeder 97 supplieselectric power to each part of face authentication machine 1. Inaddition, by connecting to power feeder 97 of face authenticationmachine 1 via a local area network (LAN) cable, power can be supplied topower feeder 97 by power over Ethernet (PoE) feeding.

In addition, power supplier 93 can receive power supply from anuninterruptible power supply device (or emergency power supply)independent of the power supply line in the entire facility. As aresult, for example, the gate device as external device 16 can performnormal operation even in a case where a power failure is generated inthe facility, and has an advantage that the security of the facility canbe maintained.

Controller 94 controls the operation of each part of external device 16.In addition, controller 94 can perform the necessary processing asexternal device 16. Controller 94 includes a processor, and is realizedby performing a program stored in a storage unit (memory) (notillustrated) by the processor.

Next, face image extraction processing will be described. FIG. 12 is anexplanatory diagram illustrating an outline of the face image extractionprocessing.

As described above, face image extractor 22 of face authenticationmachine 1 performs face image extraction processing, that is, processingof generating face image data from the captured image data of the userby each processing of face detection, face size check, and face cutout,and the face image extraction processing is similarly performed in faceimage extractor 83 of face collation server 6.

At the time of user registration, registration device 4 sends the dataof the captured image of the user to face collation server 6 via facemanagement server 5, so that face image extractor 83 of face collationserver 6 performs the face image extraction processing. In some cases,the data of the captured image of the user may be sent from managementterminal 2 to face collation server 6 via face management server 5.

On the other hand, at the time of face authentication, it is preferablethat the face image extraction processing is performed only by faceauthentication machine 1 and the face image extraction processing is notperformed by face collation server 6. In this case, face authenticationmachine 1 is provided with a highly accurate face detection function. Inaddition, face authentication machine 1 normally captures the capturingarea with camera 11, and sends face image information (captured imagedata and face frame information) to face collation server 6 at thetiming when the face is detected. As a result, the load of the faceimage extraction processing is distributed to the plurality of faceauthentication machines 1, and the load of face collation server 6 canbe reduced. In addition, since the amount of communication can bereduced, the load on the network can be reduced. Since the faceauthentication response can be performed at high speed, the faceauthentication of the target person appearing one after another can beefficiently performed.

That is, by sharing the face authentication processing of the userbetween face authentication machine 1 and face collation server 6, it isnot necessary to provide a large number of expensive authenticationmachines in which all the face authentication processing of the user isconcentrated on face authentication machine 1 as in the case in therelated art. In addition, even when the face feature data is updated,the maintenance work of the large-scale authentication machine is notrequired, and the work on face collation server 6 is sufficient.Therefore, according to the present exemplary embodiment, it is possibleto construct a face authentication system having an inexpensiveconfiguration and excellent workability.

However, face authentication machine 1 may not be provided with the faceimage extraction function, that is, face authentication machine 1 maynot be provided with face image extractor 22.

FIG. 13 is an explanatory diagram illustrating an outline of an inquiryfor face authentication using collation group information.

Face management server 5 holds information on the collation group towhich the user belongs, the association information between faceauthentication machine 1 and the collation group, and the associationinformation between face collation server 6 and the collation group. Inaddition, in collation group manager 63 of face management server 5,collation group information is generated for each face authenticationmachine 1 based on the association information between faceauthentication machine 1 and the collation group and the associationinformation between face collation server 6 and the collation group.

The collation group information is information required for faceauthentication machine 1 to request face authentication to facecollation server 6 that matches the own collation group. The collationgroup information includes identification information (group number) ofthe collation group to which face authentication machine 1 belongs anddestination information of face collation server 6 to which faceauthentication is requested from face authentication machine 1, that is,face collation server 6 corresponding to the collation group with faceauthentication machine 1. Here, the destination information isspecifically the network address (for example, IP address) of facecollation server 6, face collation server 6 that is the requestdestination for face authentication is specified by the destinationinformation, and face authentication machine 1 and face collation server6 that is the request destination for the face authentication areassociated with each other. In a case where face authentication machine1 belongs to the plurality of collation groups, the address (IP address)for each of the corresponding plurality of face collation servers 6 isincluded in the collation group information.

In face authentication machine 1, activation processor 21 acquirescollation group information from face management server 5 as operationsetting information and stores the collation group information on themaster device at the time of activation or the like. In faceauthentication machine 1, when the face of a person is detected, arequest for face collation is sent to face collation server 6 thatmatches the collation group to which the master device belongs. Therequest for face collation includes information on the collation groupto which the master device belongs.

In face authentication machine 1, the timing of acquiring the settinginformation on the collation group information and the like from facemanagement server 5 may be periodic at a predetermined timing or at apredetermined interval in addition to the time of activation, or aconfiguration in which the collation group information is distributedfrom face management server 5 may be adopted.

Face collation server 6 holds the association information between theface collation process of the master device and the collation group.When face collation server 6 receives the face authentication requestfrom face authentication machine 1, face collation server 6 specifies aface authentication process corresponding to face authentication machine1 and causes the face authentication process to perform face collation,based on the association information between the collation group and theface authentication process and the collation group obtained from faceauthentication machine 1. As a result, face collation processing isperformed in the face authentication process corresponding to thedesignated collation group.

Incidentally, in the present exemplary embodiment, in face collationserver 6, a face collation database is provided for each collationgroup, and face feature data is stored for each collation group.Therefore, at the time of face collation, the face collation may beperformed on the face feature data registered on the face collationdatabase corresponding to the collation group of face authenticationmachine 1. On the other hand, face collation server 6 may performcollation regardless of the collation group of face authenticationmachine 1, and then filter the collation result. That is, the facecollation is performed on the face feature data of all users stored inface collation server 6, and then only the collation result with theuser belonging to the collation group of face authentication machine 1may be extracted. In this case, the face collation database may not beprovided separately for each collation group.

Next, a registration operation by the user in registration device 4 willbe described. FIG. 14 is an explanatory diagram illustrating an exampleof the registration screen displayed on registration device 4.

When starting to use the facility (or in a case where the facility isnot revisited within a predetermined period after registration), theuser can perform a registration operation for authentication byregistration device 4. In the present exemplary embodiment, the user canperform the registration operation according to the guidance screendisplayed on display 42 (here, touch panel) of registration device 4. Apart of the registration operation for authentication (such as pressinga button) described later may be performed by a receptionist obtainedthe agreement of the user.

Display 42 of registration device 4 displays an initial screen on whichregistration start button 101 is displayed (refer to (A) in FIG. 14).Therefore, when the user presses (touches) registration start button101, a description related to the handling of personal information inthe face authentication system and a screen requesting the user for theagreement are displayed (refer to (B) in FIG. 14).

By pressing agree button 102 on the screen (B) in FIG. 14, the user canagree to the handling of personal information in the face authenticationsystem. As a result, registration device 4 starts acquiringauthentication information of the user. Therefore, display 42 displays ascreen (first screen) illustrating a method of acquiring specificinformation used for specifying the user (refer to (C) in FIG. 14). Theuser can stop the registration operation by pressing stop button 103.The screen for asking the user for the agreement may not require theagreement depending on the user and can be omitted. In addition, theagreement may be replaced by the operation of capturing button 104,capturing button 107, and registration button 111, which will bedescribed later. In addition, the agreement may be obtained by apredetermined procedure before using registration device 4.

In the example illustrated in (C) in FIG. 14, a screen (first screen)illustrating a method of capturing a business card of the user (here,guide for placing the business card of the user in the capturing area ofinformation acquisition camera 41B) is displayed. Therefore, when theuser places the business card on placing table 116 of registrationdevice 4 and presses capturing button 104, the business card is capturedby registration device 4. The user can stop the capturing of thebusiness card by pressing stop button 105.

When the capturing of the business card is completed, registrationdevice 4 subsequently displays a screen (first screen) illustrating amethod of capturing a face image for user registration (guidance forpositioning the face of the user in the capturing area of face capturingcamera 41A) (refer to (D) in FIG. 14). At this time, registration device4 can prompt the user to align the face by blinking capturing frame 106corresponding to the capturing area. A live image (real-time movingimage) captured by face capturing camera 41A is displayed in capturingframe 106. Therefore, when the user positions his or her face in thecapturing frame and presses capturing button 107, the capturing of theface of the user is performed by registration device 4. At this time,registration device 4 can generate a shutter sound when capturing button107 is pressed. The user can stop the capturing of the face by pressingstop button 108.

When the face capturing is completed, registration device 4 subsequentlydisplays a screen (first screen) for the user to confirm a captured faceimage (refer to (E) in FIG. 14). The captured face image (still image)is displayed on the screen, and the user presses registration button 111in a case where it is determined that there is no problem with thecaptured face image. On the other hand, the user can press re-capturingbutton 112 in a case where it is determined that there is a problem withthe captured face image. As a result, the screen illustrated in (D) inFIG. 14 is displayed on registration device 4, and the face can bere-captured. The user can stop the capturing of the face by pressingstop button 113.

When the face capturing is completed, the acquired face image andspecific information (here, including the business card image andbusiness card description information extracted from the business cardimage by OCR) are transmitted to face management server 5. In this case,at least one of the face image and the business card image may betransmitted to face management server 5 to the user who does not want toprovide both the face image and the business card image. As a result,face management server 5 performs registration processing (refer to FIG.15) related to the user. When the registration processing of facemanagement server 5 is normally completed, registration device 4displays a screen (second screen) indicating that the registration iscompleted (refer to (F) in FIG. 14 and “displaying the processing resulton the screen” in FIG. 15”). Therefore, when the user presses completionbutton 115, the registration operation in registration device 4 iscompleted. The order of acquiring the face image and the business cardimage may be reversed.

After the registration operation is completed, registration device 4 candelete the face image and the specific information stored in storageunit 44. As a result, it is possible to prevent the personal informationfrom being stored in registration device 4 used by an unspecified personfor a long period of time, and to manage the personal information of theuser more safely.

Instead of registering the user by registration device 4 describedabove, the administrator can register the user by using managementterminal 2 after preparing the face image file for user registration inadvance. In addition, the administrator can also modify theauthentication information registered by registration device 4 or addnew information from management terminal 2 in order to supplement theregistration by registration device 4.

Next, registration processing for the user performed in response to theabove-described registration operation of the user will be described.FIG. 15 is a sequence diagram illustrating a procedure of theregistration processing for the user.

When the registration operation of the user is performed in registrationdevice 4 as described above, face management server 5 starts theregistration processing for the user as illustrated in FIG. 15. At thistime, face management server 5 receives the captured image of the faceand specific information from registration device 4 as well as theoperation information of the user. In the registration processing, facemanagement server 5 first sends a request for extracting the face imageto face collation server 6 in which the target user and the collationgroup match. The request includes the captured image of the face of theuser acquired from registration device 4. At this time, in a case wherethere are a plurality of face collation servers 6 in which the targetuser and the collation group match, one face collation server 6 isselected and a request for extracting the face image is sent to one facecollation server 6.

When face collation server 6 receives the request for extracting theface image from face management server 5, face collation server 6performs face image extraction processing. In the face image extractionprocessing, the captured image of the face of the user acquired fromface management server 5 is subjected to processing such as facedetection and face cutout to extract the face image of the user. Aresponse of the face image extraction is sent to face management server5. The response includes the face image of the user.

When face management server 5 receives the response of the face imageextraction from face collation server 6, face management server 5 sendsa request for face registration to face collation server 6 in which thetarget user and the collation group match. The request can include theface image of the user acquired from specific face collation server 6.At this time, in a case where there are a plurality of face collationservers 6 in which the target user and the collation group match, therequest for face registration is sent to all (two in FIG. 15) facecollation servers 6 including the face image of the user acquired fromspecific face collation server 6.

Face management server 5 may omit the above-described request forextracting the face image to face collation server 6, and may add acaptured image of the face of the user acquired from registration device4 as well as the operation information to the request for faceregistration to face collation server 6.

When face collation server 6 receives the request for face registrationfrom face management server 5, face registration processing isperformed. In the face registration processing, a face feature amount isgenerated from the face image of the user, and the face feature amountis registered on the database. At this time, a face registration ID isassigned in association with the face feature amount of the user. Aresponse of the face registration is sent to face management server 5.The response includes a result indicating whether or not the faceregistration is normally completed and the face registration ID. Withthe completion of registration of the face feature amount on thedatabase, the face image of the user on face collation server 6 isdeleted. In addition, the face feature data may be backed up and storedin a non-volatile memory such as HDD and SSD managed separately from theface collation DB of face collation server 6.

Face management server 5 receives the response of the face registrationfrom face collation server 6, and in a case where the processing iscompleted normally, the registration processing for the user isperformed. In the registration processing, the face image of the useracquired from registration device 4 is registered on the faceinformation database. In addition, face management server 5 registersthe specific information of the user acquired from face collation server6 on the face information database in association with the face image.In addition, face management server 5 registers the face registration IDissued by face collation server 6 on the face information database asuser information.

Next, processing of face authentication of the user in faceauthentication machine 1 will be described. FIG. 16 illustrates a firstexample of the face authentication processing for the user. Here, anexample is illustrated in which a gate device (security gate) thatmanages the passage of the user is used as external device 16cooperating with face authentication machine 1.

When face authentication machine 1 detects a person's face from thecaptured image of camera 11 and acquires the face image, a request forface collation is transmitted to face collation server 6. The requestfor face collation includes the device ID of face authentication machine1 as a request source, the collation group of face authenticationmachine 1, the data of the captured image (face image forauthentication) of the target person for face authentication, the faceframe Information, and collation conditions.

When face collation server 6 receives the request for face collation,face feature amount generator 84 first generates the face feature amountof the target person from the face image of the target person acquiredfrom face authentication machine 1. Next, face feature amount collator86 of the face authentication unit corresponding to the collation groupof face authentication machine 1 collates the face feature amount of thetarget person with the face feature amount of the user registered on theface authentication database. At this time, the face collationprocessing is performed in the face authentication process in which thecollation group matches face authentication machine 1 as a requestsource. When the face collation processing is completed, face collationserver 6 transmits a response of the face collation to faceauthentication machine 1 as a request source. The response of the facecollation includes the collation result (success or failure), thecollation score, and a user code.

Face feature amount collator 86 calculates a collation score indicatingthe degree of similarity between the target person for faceauthentication and the registered user, and in a case where thecollation score is equal to or higher than a predetermined referencevalue, considers the target person of face authentication as theregistered user himself or herself to generate a collation resultindicating that the face collation is successful. On the other hand, ina case where the collation scores for all users are not equal to orhigher than the reference value, it is determined that the target personfor face authentication is not a registered user, and a collation resultindicating that face collation is failed is generated.

The face collation processing conditions (face collation parameters)performed by face collation server 6 may be added to the collationrequest. As a result, face authentication machine 1 can instruct theprocessing content of the face collation performed by face collationserver 6. For example, as a processing condition for face collation, athreshold value for the collation score is designated, and a collationresult having a collation score equal to or higher than a predeterminedthreshold value is included in the response. In addition, as aprocessing condition for face collation, the number of collation resultsis designated, and a predetermined number of collation results areincluded in the response from the higher collation score.

In addition, in face collation server 6, when the face collation iscompleted, authentication log manager 87 stores information such as thecollation result acquired by the face collation as an authentication log(history information of face authentication) on the database. At thistime, in addition to storing only the result of face authentication(success or failure) as an authentication log, the collation score maybe included in the authentication log. In addition, the face image ofthe target person acquired from face authentication machine 1 may beincluded in the authentication log. In this case, the face image of thetarget person may be encrypted and stored.

In a case where a large number of valid collation results whosecollation score exceeds the threshold value are obtained, the collationresults may be narrowed down to a predetermined number of collationresults from the higher collation score and stored as an authenticationlog. In addition, in a case where the face collation is not normallycompleted, or a valid collation result whose collation score exceeds thethreshold value cannot be obtained, only the information included in therequest from face authentication machine 1 may be stored as theauthentication log.

In addition, when face authentication machine 1 receives the response ofthe face collation indicating the success of the face authenticationfrom face collation server 6, face authentication machine 1 transmits anopen/close control signal of the door to a cooperated gate device. Thatis, face authentication machine 1 functions as a control device (gateopen/close control device) converting the response of the face collationinto the open/close control signal of the door, and then controlling theoperation of the door of the cooperated gate device. As a result, thegate device operates to open the door (that is, allow the user to passthrough) by driver 92. Face authentication machine 1 deletes the faceimage of the user stored in storage unit 14 when the transmission of thecontrol signal (open/close control signal) is completed. Here, althoughthe response of the face collation is converted into the open/closecontrol signal of the door, in a case where the open/close control ofthe door of the gate device is performed by an entry and exit managementsystem constructed as an external system, a control signal istransmitted to the entry and exit management system so that the faceauthentication result is reflected in the open/close control of the gatedevice.

FIG. 17 illustrates a second example of the face authenticationprocessing for the user. FIGS. 18A, 18B, and 18C are explanatorydiagrams illustrating an example of an authentication screen (screen atthe time of authentication) displayed on face authentication machine 1.Here, an example is illustrated in which a lock device provided on thedoor of the conference room is used as external device 16 cooperatingwith face authentication machine 1. Related to the second example of theface authentication processing, detailed description of the same itemsas in the first example described above will be omitted.

Face authentication machine 1 is installed, for example, on the wall ofa corridor facing the conference room. When face authentication machine1 detects the approach of a person by a motion sensor or the like (thatis, detects a user), face authentication machine 1 displays a standbyscreen (initial screen) of the authentication screen on display 12(refer to FIG. 18A). Here, an authentication screen (notificationscreen) displayed on display 12 includes image display area 121 thatdisplays a live image (real-time moving image) of a predeterminedcapturing area captured by camera 11, and information display area 122that displays information indicating the progress of face authenticationprocessing.

Thereafter, when the face of a person is detected from the capturedimage of camera 11 and the face image is acquired, face authenticationmachine 1 transmits a request for face collation to face collationserver 6. At this time, characters 123 and graphic 124 indicating thatthe user authentication is in progress are displayed in informationdisplay area 122 (refer to FIG. 18B). In information display area 122,the display of one of characters 123 and graphic 124 may be omitted. Inaddition, in the case of face authentication machine 1 that does nothave a display, for example, a light emitting diode (LED) lamp may beused to blink the LED when authentication is in progress and turn on theLED when the authentication succeeds to notify the progress of faceauthentication. In addition, the progress of face authentication may benotified by a display color of the LED. In addition, the display colorof the LED may be changed depending on the success or failure of theface authentication of the user.

When face collation server 6 receives the request for face collation,face collation server 6 generates a face feature amount and performs theface collation processing similarly to the first example describedabove. When the face collation processing is completed, face collationserver 6 transmits a response of the face collation to faceauthentication machine 1 as a request source.

In addition, when face authentication machine 1 receives the response ofthe face collation indicating the success of the face authenticationfrom face collation server 6, characters 127 and graphic 128 indicatingthat the user authentication is completed are displayed in informationdisplay area 122 of the authentication screen (refer to FIG. 18C). Atthe same time, face authentication machine 1 transmits a control signal(command to unlock the door) to a cooperated lock device for the door.That is, face authentication machine 1 functions as a control devicethat controls the operation of the cooperated lock device. As a result,the lock device performs an operation of unlocking (that is, permittingthe user to enter the room). Face authentication machine 1 deletes theface image of the user stored in storage unit 14 when the transmissionof the control signal is completed.

Face authentication machine 1 can be cooperated with an external devicethat only performs information processing without mechanical operationas the above-described gate device and the lock device for the door.FIG. 19 illustrates a third example of the face authenticationprocessing for the user.

Here, an example in which a payment terminal device is used as externaldevice 16 that cooperates with face authentication machine 1 isillustrated. Related to the third example of the face authenticationprocessing, detailed description of the same items as in the first orsecond example described above will be omitted.

In the third example of the face authentication processing, when faceauthentication machine 1 receives the response of the face collationindicating the success of face authentication from face collation server6, face authentication machine 1 transmits a control signal to acooperated payment terminal device (or payment system). As a result, thepayment terminal device performs processing of associating the usage feedata for the service used by the corresponding user (for example,providing meals at the restaurant) with the authenticated user, andthus, the usage fee is added to the billing data at the facility for theuser.

Next, a management work by the administrator of the face authenticationsystem will be described. FIG. 20 is an explanatory diagram illustratinga login screen displayed on management terminal 2.

On management terminal 2, when the administrator activates themanagement application and accesses face management server 5, the loginscreen is displayed. On the login screen, the administrator can inputhis or her user ID and password. When the administrator inputs the userID and password on the login screen and operates a login button, facemanagement server 5 performs user authentication. In a case where thelogin succeeds, various management screens can be displayed. In a casewhere login fails, an error is displayed.

Next, registration, reference, update, and deletion related to theassociation between face authentication machine 1 and the collationgroup in the system will be described. FIGS. 21A, 21B, and 22 areexplanatory diagrams illustrating a management screen displayed onmanagement terminal 2 at the time of registration, reference, update,and deletion related to the association between face authenticationmachine 1 and the collation group.

When management terminal 2 logs in to face management server 5, thereference screen illustrated in FIG. 21A, the registration screenillustrated in FIG. 21B, and the deletion screen illustrated in FIG. 22can be displayed at the time of registration, reference, update, anddeletion related to the association between face authentication machine1 and the collation group.

The reference screen illustrated in FIG. 21A displays a list ofassociation settings between registered face authentication machine 1and the collation group. By viewing the reference screen, theadministrator can confirm the registered contents of the collationgroup.

The reference screen is provided with list display unit 211 and faceauthentication machine designator 212. In list display unit 211, thedevice ID of each authentication machine, the collation group (number)associated with each authentication machine, and the description for thecollation group are displayed. In face authentication machine designator212, target face authentication machine 1 (device ID) can be selectedfrom pull-down menu. As a result, the display content of list displayunit 211 can be narrowed down to the specific face authenticationmachine 1, and only the collation group associated with specific faceauthentication machine 1 is displayed on list display unit 211. In acase where face authentication machine 1 is not designated by faceauthentication machine designator 212, information on all faceauthentication machines 1 is displayed on list display unit 211.

In addition, on the reference screen, items (device ID, faceauthentication machine 1, and description) can be designated and sortingcan be performed. The number of items displayed by list display unit 211(the number of face authentication machines 1) and the display range offace collation server 6 and the collation group may be designated. Inaddition, the search condition may be designated for each item (deviceID, face authentication machine 1, and description) so that the searchcan be performed.

In the example illustrated in FIG. 21A, face authentication machine 1 isgrouped according to the place where face authentication machine 1 isinstalled (for example, the location of the business establishment).

The registration screen illustrated in FIG. 21B is for setting theassociation between face authentication machine 1 and the collationgroup. Information that associates the collation group with theauthentication machine can be registered.

The registration screen is provided with authentication machinedesignator 213, collation group designator 214, and registration button215. In authentication machine designator 213, target faceauthentication machine 1 (device ID) can be selected from the pull-downmenu. In collation group designator 214, the collation group of targetface authentication machine 1 can be selected from the pull-down menu.The pull-down menu displays the collation groups already registered.When registration button 215 is operated after inputting faceauthentication machine 1 and the collation group on the registrationscreen, face management server 5 performs processing of registering thecollation group information on the input contents.

Although, in some cases, a plurality of collation groups are associatedwith one face authentication machine 1, in this case, the registrationoperation of associating one collation group with one faceauthentication machine 1 may be repeated on the registration screen. Inaddition, a plurality of collation group designators 214 may be providedon the registration screen.

The deletion screen illustrated in FIG. 22 deletes the associationsetting between face authentication machine 1 and the collation group.

The deletion screen is provided with list display unit 216, faceauthentication machine designator 217, and delete button 218. In listdisplay unit 216, the device ID of each authentication machine, thecollation group (number) associated with each face authenticationmachine 1, and the description about the collation group are displayed.In addition, list display unit 216 is provided with a check box for eachassociation setting. The association setting can be selected by thecheck box. When the registration item to be deleted is selected by thecheck box and delete button 218 is operated, processing of deleting theselected registration item is performed on face management server 5. Inface authentication machine designator 217, face authentication machine1 can be selected from the pull-down menu. As a result, the displaycontent of list display unit 216 is updated in a state of narrowing downto specific face authentication machine 1.

Here, by selecting reference on the menu screen (not illustrated), thescreen transfers to the reference screen illustrated in FIG. 21A. Inaddition, by selecting registration on the menu screen, the screentransfers to the registration screen illustrated in FIG. 21B. Inaddition, by selecting deletion on the menu screen, the screen transfersto the deletion screen illustrated in FIG. 22. In addition, by selectingupdate on the menu screen, the screen transfers to the update screen(not illustrated). In addition, when the administrator selects theassociation between face authentication machine 1 and the collationgroup on the reference screen, the screen transfers to the update screen(not illustrated).

In the example illustrated in FIG. 22, although the registeredassociations are displayed in a list on the deletion screen, thedeletion may be performed on the individual edit screen. In addition,the update screen (not illustrated) is the same as the registrationscreen (FIG. 21B).

FIG. 23 is an explanatory diagram illustrating an example of amanagement screen of the user displayed on management terminal 2.

Management terminal 2 displays the management screen of the userillustrated in FIG. 23 when managing information on the user.

The management screen of the user illustrated in FIG. 23 is for theadministrator to perform operations of registering, referencing,updating, deleting, and searching for a face image and specificinformation (here, business card information) related to the user. Theinformation (including images) input to the management screen of theuser includes the information acquired by registration device 4.

The management screen of user includes operation selection area 131,user information area 132, and business card information area 133.

In operation selection area 131, the administrator can select items tobe performed (information registration, reference, update, deletion,search, and the like). In FIG. 23, business card registration isselected as an item to be performed. As a result, the administrator cannewly input business card description information displayed on themanagement screen of the user.

User information area 132 includes a face image of the user, first andlast name, and information used for face authentication. Here, userinformation area 132 includes face image display unit 141, first andlast name input unit 142, user ID input unit 143, identification IDinput unit 144, collation group input unit 145, access permission groupinput unit 146, additional information input unit 147, activation dateinput unit 148, and invalidation date input unit 149.

The face image of the user is displayed on face image display unit 141.The administrator can delete the face image of face image display unit141 by pressing delete button 151. In a case where the face image cannotbe acquired by registration device 4, the administrator can select theprepared face image (image file) as a display target by pressing imageselection button 152. Alternatively, the administrator can display theface image selected by image selection button 152 on face image displayunit 141 instead of the face image acquired by registration device 4.The face image prepared by the administrator is not limited to theactual face image of the user, and may be a dummy image (for example,avatar image of the user).

In first and last name input unit 142, the first and last name of theuser is input. In user ID input unit 143, a user ID (for example,employee number) is input. In identification ID input unit 144, an IDthat can identify the user (for example, a series of unique numbersassigned in the order of registration of the user) is input. Incollation group input unit 145, a collation group is input. In accesspermission group input unit 146, an access permission group indicatingan administrator group who can access user information is input. Inadditional information input unit 147, additional information can beinput. In activation date input unit 148, a date on which the facecollation of the person can be valid is input. In invalidation dateinput unit 149, a date on which the face collation can be invalid isinput. By setting these activation dates and invalidation dates, it ispossible to start or end services for a plurality of users at the sametime.

Business card information area 133 includes a captured image of thebusiness card registered by registration device 4 and informationextracted from the captured image of the business card. Here, businesscard information area 133 includes business card image display unit 161,affiliation information input unit 162, activation date input unit 163,and invalidation date input unit 164.

The business card image of the user is displayed on business card imagedisplay unit 161. By pressing delete button 171 and image selectionbutton 172, the administrator can delete the business card image orselect the prepared business card image (image file), similarly to thecase of the above-described face image.

In affiliation information input unit 162, the company name to which theuser belongs, the department name of the user, the job title, thetelephone number, the fax number, the e-mail, the zip code, the address,and the like are input. In activation date input unit 163, a date onwhich the information on the business card of the person can be valid isinput. In invalidation date input unit 164, a date on which theinformation on the business card of the person can be invalid is input.

When the administrator presses registration button 167 after completingthe input to the desired input unit, the input (added or changed)information is valid.

On the management screen of the user, other specific information (forexample, information displayed on a payment card or informationindicating the place where the user stays in the facility) can bemanaged as well as the business card description information or insteadof the business card description information.

In addition, the administrator can search for necessary information byselecting a search tab (for example, collation group search) inoperation selection area 131 on the management screen of the user. Inthe search screen for such information, each input unit on theabove-described management screen of the user is displayed as a blank.For example, the administrator can search for users belonging to theinput collation group by inputting the collation group to be searchedinto blank collation group input unit 145.

When the administrator inputs search conditions with the required itemson the search screen and presses the search button (not illustrated),the screen transfers to a list display screen that reflects the searchresults (for example, screen illustrating a list of extracted users).

Next, the use of information on the user will be described. FIG. 24 isan explanatory diagram illustrating an example of a conference roomsearch screen (conference search screen) displayed as a managementscreen on management terminal 2 or the like. FIG. 25 is an explanatorydiagram illustrating an example of an attendee display screen displayedas a management screen on management terminal 2.

The conference room search screen illustrated in FIG. 24 is for theadministrator to use (reference) the information on the conferenceacquired in the face authentication processing. By accessing facemanagement server 5 from management terminal 2, the administrator canappropriately add information that cannot be acquired by faceauthentication machine 1 related to the conference held at the facility.

Information such as the event date, time, and room number (conferenceroom No.) is displayed on the conference room search screen related tothe conference held in the past in the conference room of the facility.Information on the entry history acquired by face authentication machine1 cooperated with the lock device for the door of the conference roomcan be used to generate the conference room search screen. When theadministrator selects a desired conference (click the check box of theconference illustrated at the top in FIG. 24) and presses selectionbutton 173, the screen transfers to the search result display screen inwhich the search result is reflected.

As the search result display screen, for example, as illustrated in FIG.25, an attendee display screen including an attendee list is displayedon management terminal 2.

The attendee list includes attendee information area 175 and respondentinformation area 176. In attendee information area 175, a face image andspecific information (here, the name of the company to which the userbelongs and the name of the user) of the attendee of the conference aredisplayed. In addition, in respondent information area 176, informationon the person (here, department and name) who responded to the attendee(for example, worker of a company occupying the facility) is displayed.

FIG. 26 is an explanatory diagram illustrating an example of a personalconnection search screen displayed as a management screen on managementterminal 2.

The personal connection search screen illustrated in FIG. 26 is for theadministrator to use (refer) information on the personal connections ofuser 10 acquired in the face authentication processing (here,information on the connections of people estimated from the informationon the attendee of the conference). By accessing face management server5 from management terminal 2, the administrator can appropriately addnecessary information on the personal connections of the user.

The personal connection search screen includes correlation diagramdisplay area 181 and list display area 182. In correlation diagramdisplay area 181, a diagram that visualizes the personal connectionsrelated to the person selected by the administrator is displayed. Inaddition, in list display area 182, personal connection list 185relating to the person (user 10) selected by the administrator isdisplayed. In personal connection list 185, a face image of a person,affiliation information (company name, department, and the like), anumerical value indicating the strength of the personal connection(personal connection strength), and the like are displayed.

Such a personal connection search screen may be displayed by theadministrator selecting (clicking) an area of a desired attendee on theattendee display screen of FIG. 25, for example.

The use of the conference room search screen and the personal connectionsearch screen described above is not required to be limited to those viamanagement terminal 2. For example, by granting access rights as anadministrator to a facility user (for example, worker of a companyoccupying the facility), the information processing terminal (PC,tablet, and the like) owned by the user may be able to refer to theconference room search screen and the personal connection search screen.

As described above, the exemplary embodiment is described as an exampleof the techniques disclosed in the present application. However, thetechnique in the present disclosure is not limited thereto, and can beapplied to exemplary embodiments in which changes, replacements,additions, omissions, and the like are made. In addition, it is alsopossible to combine the components described in the above exemplaryembodiment to form a new exemplary embodiment.

In recent years, in the world of Internet of Things (IoT),Cyber-Physical Systems (CPS), which is a new concept of creating newadded value by linking information between physical space and cyberspace, has been attracting attention. Under these circumstances, the CPSconcept can also be adopted in the present exemplary embodiment. Thatis, as a basic configuration of the CPS, for example, an edge devicedisposed in a physical space and a cloud server disposed in a cyberspace can be connected via a network, and face authentication processingcan be distributed and processed by a processor installed in the edgedevice and cloud server. In the face authentication processing, faceimage data of the user captured by the camera of the authenticationmachine is acquired as the edge device. Next, the cloud server performsprocessing of generating feature data from the face image data receivedfrom the edge device via the network, collates the feature data with thefeature data of the user registered in advance, performs theauthentication processing of the user, takes over the authenticationresult related to the authentication processing by the faceauthentication application software, and displays the authenticationresult on the display or the like in an output format defined on theapplication. Here, each data generated in the edge device or the cloudserver is preferably generated by a Web application or the like mountedon a standardized platform, and by using such a standardized platform,it is possible to improve the efficiency when constructing a systemincluding various sensor groups and IoT application software.

INDUSTRIAL APPLICABILITY

The face authentication system and the face authentication methodaccording to the present disclosure have the effect of easily performingthe face authentication of the user for each event that may be generatedin the facility, while safely managing the personal information of theuser who uses the facility, and are useful as a face authenticationsystem that performs face authentication processing based on image dataobtained by capturing a target person and a face authentication method.

REFERENCE MARKS IN THE DRAWINGS

-   -   1 face authentication machine    -   2 management terminal    -   3 face authentication server    -   4 registration device    -   5 face management server    -   6 face collation server    -   10 user    -   11 camera (face image acquirer)    -   12 display (display unit)    -   15 controller (processor)    -   16 external device    -   25 authentication result adjustor (history information        generator)    -   41A face capturing camera (personal information acquirer)    -   41B information acquisition camera (personal information        acquirer)    -   42 display (display unit)    -   45 controller (processor)    -   62 user manager    -   64 device manager    -   90 image generator

1. A face authentication system that registers a face image of a user ina facility and performs face authentication for each event generated dueto an action of the user in the facility, the face authentication systemcomprising: a registration device that is installed in the facility andacquires a first face image for registration of the user and specificinformation different than the first face image, the specificinformation being used for specifying the user; at least one camera thatacquires a second face image for authentication of the user, the secondface image being captured when the event is generated; a face managementserver that is connected to the registration device via a network andstores the first face image for registration in association with thespecific information; and a face collation server that generates facefeature data from the first face image for registration and the secondface image for authentication, and performs collation processing betweenpieces of the face feature data.
 2. The face authentication system ofclaim 1, wherein the event is generated by an action of passing, by theuser, through a gate device that manages passage in the facility, andthe at least one camera cooperates with the gate device.
 3. The faceauthentication system of claim 1, wherein the event is generated by anaction of passing, by the user, through a door that manages an entryinto a predetermined room in the facility, and the at least one cameracooperates with a lock device for the door.
 4. The face authenticationsystem of claim 1, wherein the event is generated due to an action ofpayment, by the user, via a payment terminal device used in thefacility, and the at least one camera cooperates with the paymentterminal device.
 5. The face authentication system of claim 1, whereinthe registration device acquires information displayed on a businesscard possessed by the user as the specific information.
 6. The faceauthentication system of claim 1, wherein the registration deviceacquires information displayed on a payment card possessed by the useras the specific information.
 7. The face authentication system of claim1, wherein the registration device acquires information indicating aplace of stay of the user in the facility as the specific information.8. A face authentication method that registers a face image of a user ina facility and performs face authentication for each event generated dueto an action of the user in the facility, the face authentication methodcomprising: acquiring a first face image for registration of the userand specific information different than the first face image, thespecific information being used for specifying the user in the facility;acquiring a second face image for authentication of the user, the secondface image being captured when the event is generated; storing the firstface image for registration in association with the specific informationin a face management server; and generating face feature data from thefirst face image for registration and the second face image forauthentication, and performing collation processing between pieces ofthe face feature data by a face collation server.
 9. The faceauthentication system of claim 1, wherein the second face image of theuser is detected from an image captured by the at least one camera, cutout from the captured image, and image data of only the second faceimage, from among the captured image, is transmit to the face collationserver.
 10. The face authentication system of claim 9, wherein the facecollation server is physically separate from the at least one camera,and the image data of only the second face image is transmit to the facecollation server to reduce a load on the network.
 11. The faceauthentication system of claim 1, wherein the second face image of theuser is detected from an image captured by the at least one camera, cutout from the captured image, and image data of the second face image andof a predetermined capturing area is transmit to the face collationserver.
 12. The face authentication system of claim 1, wherein theregistration device transmits the first face image for registration ofthe user and the specific information to the face management server, anddeletes the first face image for registration of the user and thespecific information after transmission to the face management serverwhereby the first face image for registration of the user and thespecific information are not stored in the registration device.